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Abstract 



Although the National Targets for Education and Training in England and Wales include 
indicators for lifelong learning, and the progress towards the targets set for these indicators has 
been lauded by politicians and other observers, much of this apparent progress is actually 
accounted for by changes in these same indicators at Foundation level. However, once the 
'conveyor belt effect', of passing increasingly qualified 16-18 year-olds into the working-age 
population instead of less qualified 60 and 65 year-olds, is taken into account then progress in 
qualifying those of working-age is much less. In fact, there is then very limited evidence that 
Lifelong Learning targets have had any impact at all. Certainly work-based training has not 
increased, and may even have declined over the last decade, while some socio-economic 
inequalities in adult participation in education and training have worsened. The paper examines 
this apparent weakness for the current approach and its implications for the measurement of 
progress, in the light of a research review carried out for the National Assembly of Wales. 



Introduction 

The use of attainment targets in education is by no means a new phenomenon, dating back at least 
to the nineteenth century school-based practice of 'payment by results' (Marsden 1991). More 
recently, the burgeoning use of target setting in education has been a key feature of what Neave 
(1988) referred to as the 'rise of the Evaluative State', with performance indicators and 
'accountability' a key feature in most public and private sector policy-making. The present concern 
with measuring educational 'quality' can clearly be located within broader concerns for global 
economic and structural changes over the last three decades (Bagnall 1994, Smyth and Dow 
1998). Thus, it is of little surprise that countries are now beginning to devise targets specifically 



1 



BEST COPY AVAILABLE 



for lifelong learning, with education's renewed role as what the Prime Minister has described as 
'the best economic policy we have'. The prominence of accountability-based target setting has 
been rising steadily in post-compulsory education since the 1980s; most notably with the 
established trend in higher education systems towards performance-related budgeting (Cave et al. 
1997, Layzell 1998). More recently, the use of attainment targets has spread into the broader area 
of lifelong learning, with a number of countries recognising the need for introducing targets for 
participation and achievement in vocational and non- vocational learning across adult populations. 
A coherent and strategic drive towards a comprehensive nation-wide system of educational 
targets began to emerge in the UK from the end of the 1980s and the setting of the National 
Targets for Education and Training (NETTs). The NETTs were first suggested in the 1989 CBI 
document Towards a Skills Revolution, which focused on reversing the perceived characteristics 
of the British workforce as 'under-educated, under-trained and under qualified' (CBI 1989). The 
policy of target-setting is now defined as 'taking action by setting specific goals and targets 
designed to raise educational standards' (DflEE 1997, p.5). 

Two years after the 1991 launch of the National Targets, an independent employer-led body - the 
National Advisory Council for Education and Training Targets (NACETT) - was founded to 
oversee the NETTs, charged with monitoring progress towards achieving the targets, advising the 
government and 'providing business leadership'. The Targets were subsequently revised in 1995, 
introducing an additional target for the attainment of higher education qualifications, as well as 
increasing the proportions expected to attain at least a level 3 qualification (NACETT 1995). In 
1997, NACETT proposed a second review, resulting in England, Wales, Scotland and Northern 
Ireland being given responsibility for setting their own targets alongside overarching targets for 
the UK as a whole (DfEE 1997, NACETT 1998). In Wales, this task was given to the Education 
and Training Action Group for Wales (ETAG). ETAG's resulting Education and Training Plan for 
Wales (1999) proposed a range of outcome targets for both pre- and post- 16 education and 
training in Wales, building on previous sets of targets suggested in A Bright Future: the way 
forward (Welsh Office 1995a), A Bright Future: Beating the Previous Best (Welsh Office 1997) 
and Learning is for Everyone (Welsh Office 1998). In particular, the following qualification-based 
Lifelong Learning targets were recommended in the 1999 ETAG Plan: 

• The proportion of adults of working age without qualifications to reduce from some 1 in 4 in 
1996 to 1 in 7 by 2002 and to fewer than 1 in 8 by 2004 

• The proportion of adults of working age with an NVQ level 2 or equivalent to increase from 
over 5 in 10 in 1996 to 7 in 10 by 2002 and over 7 in 10 by 2004 
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• The proportion of adults of working age with an NVQ level 3 or equivalent to increase from 
some 3 in 10 in 1996 to approaching 5 in 10 by 2002 and over 5 in 10 by 2004 

• The proportion of adults of working age with an NVQ level 4 or equivalent to increase from 
some 1 in 5 in 1996 to over 1 in 4 by 2002 and approaching 3 in 10 by 2004 

It should be stressed that the clear intent expressed in all these recent policy documents relating to 
lifelong learning is to produce an amelioration both of the skills and the life chances of those 
individuals already in the workforce or of working-age - of 'transforming learning in the 
workplace... because productivity depends on the whole workforce we must invest in everyone' 
(DfEE 1998a, p.ll, see also DfEE 1998b, Welsh Office 1998, ETAG 1999). The importance of 
stressing that this is the purpose of the policy will become clear later in the paper. 

Target setting in education and training is by no means an exact science. Nevertheless, from our 
review of academic and professional literature (Gorard et al. 1999a), the following recurring areas 
have been highlighted as principal areas for consideration in setting 'high quality' targets (see 
Williamson et al. 1992, Nuttall 1994, Tymms 1995, Donald and Denison 1996, Loveman 1997): 

• Usefulness to Policy-Makers 

• Relevance and Endurance 

• Positioning Targets as Part of Wider Educational Reform 

• Practitioner Acceptance 

• Evaluation of the processes which deliver targets 

Although these diverse criteria highlight the complexity of educational target setting, it is 
nevertheless possible to draw together a framework for high-quality education targets. At a simple 
level, the DfEE's (1997) model of SMART targets (Specific, Measurable, Achievable, Realistic 
and Time-related) provides a broad guideline. The logic of target-setting is that policy-makers can 
specify what is important, that 'this' can be achieved, that this can be measured, and therefore that 
this can be shown to have been achieved (Gann 1999). We have written elsewhere about issues of 
importance and relevance, so this paper concentrates on issues of measurement and the accuracy 
of the information required for a successful target -setting process. 
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Methodological issues 



Sources 

Progress towards each target for lifelong learning is assessed for this paper via secondary analysis. 
The data come chiefly from the national results of statutory assessment in Wales provided on 
spreadsheet by the Welsh Joint Education Committee (WJEC), the Labour Force Survey as 
supplied on spreadsheet from the Office of the National Assembly of Wales, and via the National 
On-line Manpower Information System. At first sight, many other sources are available which 
give an estimate of the frequency of lifelong target indicators in the population, but, on 
consideration, most sources suffer from defects and none is clearly preferable overall to National 
Statistics combined with the quarterly Labour Force Survey. For example, one-off surveys such as 
Future Skills Wales, Basic Skills, and the National Adult Learning Survey can be used to calibrate 
other sources, but are of little use in either predicting or measuring trends over time. Some 
sources, such as records of MAS, NVQs or even FE participation, provide only a small part of the 
required picture, and cumulating from these involves the obvious dangers of bias through 
omission and double-counting. In some cases, the keeping of quality records on recent initiatives, 
such as NVQ or CSI, is so new that they are also only useful as 'snapshots' at present and cannot 
be used to detect trends. According to Schagen et al. (1997), the Youth Cohort Study has only 
one or two useful cohorts; the GNVQ database holds registrations but not outcomes; and the 
NISVQ and NCVQ databases hold records of qualifications awarded but not individuals. In 
several cases, survey responses need to be treated carefully for technical reasons. For example, 
the Future Skills Wales (1998) study reported a response rate for employers of 45%. The study 
drew a sample of 29,951 employers, but only 5,790 (or 19%) of these were interviewed, with the 
remainder being unobtainable, refusing to participate or excluded because of quota restrictions. In 
producing the response rate of 45 per cent, the study was nevertheless unable to collect data from 
an additional 1,866 employers whose telephone number was unobtainable, engaged or not 
recognised; and the 1,222 who did not keep appointments, for example. All of these omissions 
may introduce bias into the sample, allowing the survey to over-represent larger, more successful 
employers (who are less likely to move, or be cut off by BT, or who employ several telephonists), 
and these are more likely to report training. 

Despite its own shortcomings as a measure of targets, the LFS therefore remains the major source 
of data. It is preferred since it involves households rather than institutions or employers, and is 
face-to-face rather than postal (Welsh Office 1993). It uses a rolling sample (80% repeated) for 
the UK of around 150,000 in 60,000 homes every quarter. A repeated survey of this size gives a 
reasonably reliable estimate of changes in the qualifications rates of the population. It has been 
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helpful also to have access to the intermittent boosted LFSs, organised by the Welsh Office in 
conjunction with the TECs (Welsh Office 1994, 1995b, Education and Training Statistics 1997). 

Indicators 

All indicators used in the measurable targets are 'filtered' through a range of processes which 
affect their reliability, from inconsistency in definition, variations of concept in time and space, 
divergence in data collection procedures, and errors in transcription and processing (Johnstone 
1981). Doubts have always existed over the reliability of examinations leading to qualifications 
with the same title, and whether these are comparable over time and place (Kitchen 1999, Cassidy 
1999a, Cassidy 2000). Britain is probably unique among OECD countries in using different 
regional authorities (local examination boards) to examine what are meant to be national 
assessments at 16+ and 18+ (Noah and Eckstein 1992). This raises an issue of whether the same 
qualification is equivalent between the boards in terms of difficulty (and considering that the UK 
currently has around 17,500 publicly funded qualifications from 250 awarding bodies). The large- 
scale use of vocational qualifications is relatively recent, and before 1997 many published figures 
do not include them at all. Thus, vocational qualifications were either ignored as too few to be 
included or else assumptions were made on the basis of very scanty evidence (NACETT 1995). It 
is already clear that qualifications with the same title are not even equivalent in terms of subject 
content, as each board sets its own syllabus. Nor are they equivalent in the form of assessment, or 
the weighting between components such as coursework and multiple-choice. Similarly, there is no 
evidence that the different subjects added together to form aggregate benchmarks are equivalent 
in difficulty to each other; yet the standard GCSE benchmark (passes at A*-C grades) gives the 
same value to an A* in Music, a B in Physics, and a C grade in Sociology. Nor is there evidence 
that qualifications with the same name are equally difficult from year to year. In fact, 
comparability can be considered between boards in any subject, the years in a subject/board 
combination, the subjects in one board, and the alternative syllabuses in any board and subject. All 
of these are very difficult to determine, especially as exams are neither accurate nor particularly 
reliable in what they measure (Nuttall 1979). Pencil-and-paper tests have little generalisable 
validity (i.e., in predicting the performance of other tasks), and their link to other measures, such 
as those of occupational competence, is also very small (Nuttall 1987). There are often reports of 
errors in marking and grading assessments (e.g. Cassidy 1999b, Cassidy 1999c). The fact is that 
even the narrow version of propositional knowledge tested by examinations is very difficult to 
assess. 

The equivalence between different qualifications, such as is used to define the 'levels' of 
equivalence to NVQs for targets, requires judgement and some guesswork. Robinson (1995) 
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described some of the ensuing decisions as 'arbitrary' and having little meaning in many cases, 
being based on a supposed level of desirability or prestige, rather than utility or labour market 
returns. As in any survey, some respondents reply 'don't know' or 'no answer' when faced with a 
question about their highest qualification. If these responses are assumed to come equally from all 
possible responses, and are divided proportionately between the remaining categories, the likely 
outcome is a bias towards higher qualifications. This is because a contrary assumption would be 
that those with PhDs, or postgraduate certificates, are more likely to respond and more likely to 
know what their qualifications are than those with level 1 or no recognised qualifications. For the 
same reason, if these responses are simply ignored, then the same effect results, since if they had 
answered the 'don't knows' might have been more common among the lower-level qualifications. 
If, on the other hand, the assumption is made that 'no response' is equivalent to 'no qualification', 
the overall population estimates will be biased towards the lower level of qualification, since at 
least some of the 'no responses' will actually have qualifications. There is no clear basis on which 
to partition the null responses between these three alternatives. Perhaps the safest assumption is 
that null responses all represent qualifications below level 2, and should be partitioned between 
none and level 1 in proportion to the existing frequency of those categories. Official sources 
currently allocate them proportionately to all other categories (so that at least some of those who 
do not know are 'awarded' a PhD). 

Similar comments can be made about the much larger group (7% of base) classified in the LFS as 
'other qualifications' (i.e., not one of the other named 46 qualification categories). The 
Department for Education and Employment (DfEE) and Welsh Office assumption has been that 
10% of these qualifications are at level 3, 35% are at level 2, and the remainder at level 1. Since 
this group is so large, this decision makes a significant difference to higher-level Targets 
compared to assuming that most 'other qualifications' are at level 1 . In fact, it is hard to imagine 
many qualifications, other than those from overseas (which are anyway at least partly covered by 
'A level or equivalent', etc.), which would be of 2 A-level standard. Scottish CSYS and Scottish 
Highers or equivalent are both explicitly covered, and an assumption is generally made that two- 
thirds of these are at level 3, and the other third at level 2. Trade apprenticeships (8% of base) are 
generally divided into half level 2 and half level 3. As can be imagined, there is very little direct 
empirical justification for any of these assumptions, and yet their importance cannot be 
overestimated, since they affect two of the most common responses. Therefore perhaps as many 
as 20-25% of cases in the LFS are somewhat arbitrarily allocated to an NVQ equivalent. The 
potential impact can be seen in Table 1 which shows the proportion of the working-age 
population with level 2 and 3 qualifications according to the LFS. In each case, the first row uses 
all operational governmental assumptions about non-response, other qualifications, trade 
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apprenticeships, and Scottish qualifications. The second row treats other qualifications as level 1, 
and non-response as level 1 at best. As can be seen, the differences are significant (although 
perhaps declining over time). 



Table 1 - Testing the impact of assumptions 





1996 


1997 


1998 


1999 


Level 2a (WO) 


54.1 


55.9 


58.9 


59.9 


Level 2b 


48.2 


52.0 


55.4 


56.1 


Level 3a (WO) 


32.0 


33.1 


36.7 


36.4 


Level 3 b 


29.8 


32.1 


35.8 


35.5 



Other issues of measurement 

A clear issue relating to the assessment of trends is the consequence of changes in the definition of 
data or in the completeness of collection, and the problems that result for reliability (Johnstone 
1981, FEFCW 1998). Even apparently minor inconsistencies, such as varying the day of an annual 
census, can have profound impacts. Some targets, while ostensibly similar, have in feet changed 
over time. In 1993 the Foundation Target 1 for Level 2 qualifications was changed from a 
requirement of 4 GCSEs grade A-C (NACETT 1993) to 5 GCSEs grade A*-C (National Targets 
Brief 1994). From Spring 1993, the LFS used an expanded version of the variable covering an 
individual's highest qualification. More vocational options were added, the level 2 criterion was 
changed from four GCSEs A-C to five GCSEs A*-C, and those with A levels were asked whether 
they had two or more, as a criterion for level 3 (National Targets Brief 1994). As a consequence, 
previous figures are no longer directly comparable with more recent ones, restricting the length of 
time for which trends can be assessed. A similar but smaller change had taken place in 1992. In 
1995 there was a slight adjustment (up) of figures following reweighting from OPCS (NACETT 
1995). Changes in Spring 1996 mean that the figures for level 2 are now a little lower than they 
would have been (NACETT 1998). These changes have all made comparisons over time more 
difficult. 

There are large seasonal variations in some indicators of education or training, so the time in the 
year when a sample is taken could make a big difference to the population estimate (National 
Targets Brief 1994). From 1993, the LFS became seasonal rather than annual (NACETT 1995), 
and according to NACETT (1994), until 1994, the Spring figures from the LFS were used to 
assess progress, but from 1994 the Autumn figures were used (at the same time as the questions 
on qualifications were changed). 



The Welsh Office (1993) specified that calculation of the achievement of any population was 
based on an individual's place of work or study, rather than their home location. A majority of 
students at Cardiff University, for example, have their 'permanent residence' in England or further 
afield and return there on completion of their course, yet they are included in the measurement of 
indicators for Wales. If Cardiff University were to expand or to take an even larger proportion of 
undergraduates from England, then progress towards the target for Level 3 qualifications would 
be boosted without necessarily adding a single qualification to the actual residents of Wales. 
However their population is defined, local targets, by their nature, take no account of the 
increasing geographic mobility of labour which is most marked among those with the highest 
qualifications (for example, Gorard et al. 1999b). Perhaps one indication of this consideration is 
that 'local' targets are themselves problematic by their very nature in an era of mobility and 
perceived globalisation. 



Progress towards the targets 

• The proportion of adults of working age without qualifications to reduce from some 1 in 4 in 
1996 to 1 in 7 by 2002 and to fewer than 1 in 8 by 2004. 

This Target is an extension to 2004 of the Targets for 2002 set out in Learning is for Everyone 
(April 1998), but with the alteration of the definition of adults from those aged 19 years or over to 
those of working-age. This change has the effect of bringing the Target for adults closer to 
attainment, since the revised version excludes those adults past working-age, who are generally 
less well-qualified than average; and includes those aged 16-18 who are generally better qualified 
than average. There must be considerable doubt whether, despite the reduction, the Target can be 
attained by 2002. Hence, although Figure 1 also suggests a significantly decreasing trend (based 
on the 'optimistic' official assumptions described above), it is not one that will meet the Target of 
14% by 2002. 
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Figure 1 - Percentage of working age population with no qualification 



No qualification 




[Spring season, from LFS data supplied by the Office of the National Assembly] 

Lack of qualifications generally increases in frequency with age, especially for women. This is, in 
fact, contrary to what might be expected for a lifelong indicator, since the longer an individual has 
been in the workforce, the greater the number of opportunities they should have had for further 
education or training. The implication is that very few additional qualifications have traditionally 
been gained after initial education. 

• The proportion of adults of working age with an NVQ level 2 or equivalent to increase from 
over 5 in 10 in 1996 to 7 in 10 by 2002 and over 7 in 10 by 2004. 

The Level 2 Target appears attainable by 2002 using the official assumptions for calculation 
(Figure 2) 
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Figure 2 - Percentage of working age population with level 2 qualifications 



Level 2 




[Spring season, from LFS data supplied by the Office of the National Assembly] 

• The proportion of adults of working age with an NVQ level 3 or equivalent to increase from 
some 3 in 10 in 1 996 to approaching 5 in 10 by 2002 and over 5 in 10 by 2004. 

There has been significant growth towards the Level 3 Target, but despite previous downwards 
adjustment, the figure for attainment is still a long way short of the proportions envisaged even 
for 2004 (Figure 3). 
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Figure 3 - Percentage of working age population with level 3 qualifications 



Level 3 



50 
45 
40 
35 
30 
25 
20 
15 
10 
5 
0 

[calculated from LFS data supplied via NOMIS] 

• The proportion of adults of working age with an NVQ level 4 or equivalent to increase from 
some 1 in 5 in 1996 to over 1 in 4 by 2002 and approaching 3 in 10 by 2004. 

From the figures available, progress towards the Level 4 Target seems erratic. Although on one 
interpretation the 2002 target is attainable (Figure 4), the indicator remains at around 1 in 5 (20 
per cent), and whilst there has been some growth, it has recently been much slower than would be 
required to meet this Target. 
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Figure 4 - Percentage of working age population with level 4 qualifications 



Level 4 




[calculated from LFS data supplied via NOMIS] 



Conveyor belt 

In summary, progress towards the targets is uneven using out best estimates, and generally less 
than that necessary to reach the targets. More disturbingly, the progress that is evident is largely 
dependent on progress towards the Foundation Targets. 

Jagger et al. (1996) estimated from the qualification rate of new entrants to higher education (HE) 
that there will be a 15-20% growth of graduates from 1996 to 2000. There would, therefore, be 
an annual flow of 210,000 graduates into the workforce. On the basis of this, they conclude that 
the growth of qualifications among those already in the workforce is so negligible that it can be 
ignored in their model as being less than the error component in their estimate of the number of 
graduates. An extension of this idea is what we have termed here the 'conveyor belt' effect 
distorting progress towards the National Targets. The Foundation (school) and Lifelong Targets 
are not symmetrically related. Changes in the qualification indicators for schools will, in time, 
produce changes in the equivalent qualification indicators for the adult population; but clearly the 
reverse is not true. Thus, at least some of the changes in the lifelong indicators can always be 



traced back to foundation indicators, and it is important both for setting targets and assessing 
progress towards them that an estimate of this 'conveyor belt' effect is produced. 

In order to estimate the size of this effect and partition any progress towards Lifelong Targets 
into that produced by initial education and that caused by increasing levels of learning among 
adults, it is important to have figures for the size and qualification rates of every age-group. The 
proportion of school-leavers with any level of qualification up to level 3 is higher than in the 
population as a whole and is rising every year (Table 2). Moreover, the proportion with any level 
of qualification is lowest amongst the older age groups (Table 3). Among the employed, 
attainment of Level 4 qualifications for example increases with age cohort from 16 to 29, peaking 
before age 49 and dropping to around 19% for those of retirement age. Therefore, significant 
progress towards the Lifelong Targets would be achieved without a single extra adult 
participating in education or training and gaining qualifications, simply as a consequence of adding 
qualified school-leavers to the adult population. 



Table 2 - The results of school-leavers in Wales 





1993 


1994 


1995 


1996 


1997 


1998 


# 15-year-olds 


32051 


32323 


35842 


37040 


35868 


35518 


No GCSEs 


12% 


10% 


10% 


10% 


9.9% 


9.3% 


5 GCSEs A*-C 


37% 


39% 


41% 


42% 


44% 


46% 


# 17-year-olds 


34900 


34300 


33500 


34700 


38000 


37600 


2+ A-levels 


18.7% 


19.5% 


18.9% 


19.3% 


23.4% 


20.6% 


Advanced GNVQ 


- 


- 


- 


- 


3.6% 


6.1% 



[calculated from WJEC data] 



Table 3 - Percentage employed with L4 qualifications by age/gender: UK 1995 



Age 


Male 


Female 


16-19 


0.9 


0.9 


20-24 


17.6 


18.6 


25-29 


25.5 


28.1 


30-39 


27.9 


27.6 


40-49 


27.9 


24.3 


50-59 


21.5 


19.1 


60+ 


18.9 


- 



[Jagger et al. 1996] 
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The LFS suggests that some 44% of females aged 59 and 34% of males aged 64 have no 
qualifications; while 16% of females aged 16 and 20% of males aged 16 have no qualifications. If 
it is assumed that the approximate cohort size of any year group is 40,000, the gender split is 
approximately 50:50, and that men work for 48 years and women 43 years on average, then the 
working-age population of Wales is 48*20,000 + 43*20,000 (or 1,820,000). Every year, 44% of 
20,000 and 34% of 20,000 (or 7,800) unqualified people leave the work-force on retirement, and 
16% of 20,000 plus 20% of 20,000 (or 3,500) unqualified people join it. If in 1998, 20% (or 
364,000) of the workforce had no qualifications, then in 1999 the figure would be 19.7% 
(364,000 - 4,300). 

Similarly, if it is assumed that every year from 1993, 40% of the 15-year-old cohort gain 5+ 
GCSEs, then by 1999 six cohorts totalling 240,000 join the working-age population, of whom at 
least 96,000 have NVQ Level 2 or equivalent qualifications. If it is further assumed that 15% of 
the 59/64-year-old cohort have Level 2 qualifications, then the effects of their retirement between 
1993 and 1999 will be to remove sue cohorts, again totalling 240,00, from the working-age 
population. However, of these only 36,000 have level 2 qualifications. Therefore, on the 
assumption that in 1993, 50% of the working-age population of 1,820,000 had qualifications to 
NVQ Level 2 or equivalent (910,000), by 1999, 910,000 plus 60,000 (96,000 - 36,000) would 
have NVQ Level 2 equivalent. This growth to 53% of the working-age population would be 
achieved without any improvement in terms of qualifying adults at all, but simply as a result of 
adding qualified school-leavers to the working-age population. Hence, progress in terms of 
increasing qualifications amongst the adult population (i.e., non-school-leavers) is confined to the 
residual that remains once the 'conveyor belt' effect is taken into account. 

Again, if every year from 1993, 20% of the 17-year-old cohort gain 2+ A-levels, then by 1999 six 
cohorts, totalling 240,000, join the working-age population. Of these, at least 48,000 have NVQ 
Level 3 qualifications. On the basis of a total working-age population in Wales of 1,820,000, and 
assuming that 10% of the 59/64-year-old cohort are qualified to NVQ Level 3 or equivalent, as 
these 1 8,200 individuals go into retirement over the six-year period, the impacts are as follows. In 
1993, 29 % of 1,820,000 had NVQ Level 3 or equivalent (527,800). Thus, by 1999, 527,800 plus 
29,800 (48,000 - 29,800) will have attained this level of qualifications, a growth to 31% of the 
working-age population, again without any improvement in terms of qualifying adults at all. To 
some extent, therefore, the Lifelong Targets are not actually 'lifelong', but are partly an extension 
of the long-term impact of targets for initial education. 
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To sum up - this kind of progress, requiring no actual amelioration of the position of those 
already of working -age is almost of the order of magnitude demanded by the targets, and certainly 
of the order of magnitude of the actual progress. 



The impact of targets 

It is therefore possible to question the actual impact of Targets on the growth of related indicators 
(for the benefit of those who believe that Targets are stimulating rather than simply measuring 
that growth), and to estimate this impact by comparing progress before and after the creation of 
the NETTs in 1991. Although it is clear that other factors than the Targets also differ pre- and 
post- 1991, it is certainly not clear that these policies and interventions since 1991 have been any 
less, or any more effective (Gann 1999). Therefore, it is fair, in the first instance, to assume that if 
the policy of setting Targets has been a useful one, then this will be reflected in improved progress 
in terms of levels of qualifications since the Targets were introduced. For example, NACETT 
(1994) have claimed that 'We have made real progress since the National Targets were launched 
in 199 T, but without making clear against what criterion this progress was measured. Since they 
present no analysis of the status ante, the claim - like most subsequent commentaries - suffers 
from what Huff (1991) calls the 'missing comparator'. NACETT were formed in 1993, and it was 
only then that Targets began to become known to employers and educators (a process which is 
still not complete), and policy changes are anyway generally slow to register any impact and are 
incremental in effect. In this context it may be noteworthy that the lowest annual growth in the 
indicator for the original Lifelong Target 3 is in 1996 (actually negative), and the growth in 1998 
is otherwise the lowest since 1991 (Table 4). The year of peak growth is 1991/1992, when the 
targets were first devised (although this is almost certainly a coincidence). The average growth 
since 1992 has been 7.5% per year, and the average growth until 1992 was 8.7%. Overall then, it 
is likely that not much has changed as a result of target-setting. 



Table 4 - Percentage of 21-year-olds with level 3 qualifications: Britain 





1988 


1989 


1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 


L3 


27.5 


28.6 


30.5 


30.7 


34.7 


37.6 


40.9 


45.0 


44.3 


48.8 


50.3 


change 


- 


4 


7 


1 


13 


8 


9 


10 


-2 


10 


3 



[NACETT 1998] 
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Although the only available figures over a long period of time are from Britain, it is clear that 
there has been a marked trend towards improved qualifications since 1988 (i.e. before the setting 
of targets). There is no particular pattern to the proportionate annual increases in any indicator, 
but there is some evidence that the strength of the growth trend is declining over time (Table 5). 
It is perhaps no surprise, given the evidence of the conveyor belt effect, that Lifelong Learning 
Targets do not appear to have led to any improvement at all on preceding progress. To some 
extent they may actually have led to an increased concentration on what we have termed 
elsewhere 'front- loaded' provision (Gorard et al. 1999b). 



Table 5 - Percentage of 19-year-olds with level 2 qualifications: Britain 





1988 


1989 


1990 


1991 


1992 


1993 


1994 


1995 


1996 


1997 


1998 


L2 


48.4 


50.6 


52.0 


54.5 


59.1 


62.3 


66.0 


66.9 


68.4 


71.3 


72.4 


change 


- 


5 


3 


5 


8 


5 


6 


1 


2 


4 


2 



[NACETT 1998] 



Conclusions 

In many respects the conclusions of this paper make depressing reading. Progress towards several 
of the National Targets for Lifelong Learning is slow in Wales (and in other home countries 
where accurate figures have been made available, Gorard et al. 1999a). Much of this progress 
actually stems from growth in school-based qualifications, and there is little suggestion the 
qualification of adults while they are adults has improved much since 1991. The progress that has 
taken place appears to be unaffected by the setting or monitoring of targets. This is all in direct 
contradiction to the proclaimed purposes of the policy of target-setting for lifelong learning, 
despite reports of policy success. Our chief finding here, based on secondary analysis of figures 
for all Wales, co nfir ms our previous suggestions, based on a large-scale survey in South Wales 
(e.g. Gorard et al. 1999b), that work-based training is not rising to meet the challenge of lifelong 
learning, and that employers are supporting an ever decreasing proportion of training. If it true 
that 'in the learning age individuals and their employers will share a responsibility for increasing 
the quality and quantity of learning at work' (DfEE 1998a), then that 'learning age' is certainly no 
closer to realisation as a result of the National Targets. 

On the other hand, the ETAG Targets miss out a great deal of the learning which actually occurs 
amongst the adult population, in consequence of their focus upon certificated education and 
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training (Gorard et al. 1999c). In part, of course, this reflects the paucity of data (only the 
National Adult Learning Survey 1997, Beinart and Smith 1998, provides systematic data at the 
national level on wider patterns of educational participation amongst adults and this provides only 
a snap-shot), but is also a consequence of the Targets themselves which by their current nature 
privilege the apparently measurable. 

Despite this privileging there are serious difficulties in actually estimating progress towards the 
'measurable' targets. The calculations on which the recommendations in other sections are based 
use the standard Welsh Office/DffiE assumptions about problematic qualifications. It has been 
shown in this paper that these assumptions are important but perhaps now rather dated. Over 50 
qualifications are now listed in the LFS analysis (many of which have very few cases), but there 
are still large numbers of 'other qualifications'. It would be interesting to conduct further research 
into the nature of these. 

If we retain but reconsider the idea of targets for lifelong learning there are some additional issues 
to consider. For example, there is no strong rationale for the continued exclusion of individuals 
past the conventional retirement age from the Targets, although again data are not readily 
available. It is clear that, in addition to being exclusionary, the over-emphasis on qualifications for 
those of working-age is leading to an over-education paradox whereby many people are employed 
in posts not requiring their actual level of qualification (Slater 2000), such that qualification and 
lack of qualification is a poor guide to competence (Fuller and Unwin 1999). There may also have 
to be revision of the analytical assumption that men retire at 65 and women at 60. Planning should 
be underway for this already, and the new set of assumptions underlying target revisions should be 
based on the new non-gendered definition of retirement age. In addition it would be useful to 
consider ways of rephrasing some Targets so that they are expressed as average qualification (or 
participation) per resident, rather than a proportion meeting a certain threshold (Tymms and Stout 
1999). As currently designed, for example, if an educator and a student work hard together to 
obtain three GCSE passes rather than two, there is no impact on progress towards achieving 
Targets. If the Targets have an impact on qualifications and participation, the current set appear 
to encourage a focus on those who are on the 'cusp' (for example, between grades D and C at 
GCSE). Work in the USA suggests that the setting of thresholds serves progressively to exclude 
those furthest from it. A more carefully designed 'average' target could allow all residents of 
Wales to be included in progress towards meeting the Targets. 
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